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(54) Link reservation in communications networks 

(57) An efficient distributed means for setting up, 
taking down, and updating connections in a communi- 
cations network, which is robust in the presence of fail- 
ures. More specifically with this invention a path for a 
connection is computed and reservation requests are 
simultaneously sent through separate channels to each 
of a set of nodes of the path for the connection to reserve 
and determine if resources are available for the connec- 
tion. Upon acknowledgement that the links for the con- 
nection are available and have been reserved, a setup 
message is sequentially transmitted between the adja- 
cent nodes along the path until the setup messages 
reaches the destination node. The switches in each 
node then configure themselves in response to the set- 
up message so as to establish the connection. 
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Description 

The present invention relates to communication 
networks. 

Optical wavelength division multiplexing (WDM) s 
networks can provide enormous capacities (several 
hundred gigabits per second) to a large number of net- 
work users. In WDM the fiber bandwidth is split into mul- 
tiple channels, each operating at a different optical 
wavelength. Each wavelength can carry data at gigabit- 
per-second rates. Such a network is shown in Figure 1 . 
The network consists of nodes (Nodes A-E) intercon- 
nected by fiber-optic links (106) in an arbitrary mesh to- 
pology. Each link carries traffic at several wavelengths. 
Each wavelength can be switched and routed independ- 
ently ol the others at each node. The network provides 
lighipaths (connections). A path is a set of links in the 
network that interconnect two nodes in the network. A 
lightpath is established on a particular wavelength and 
path through tne network. In general a connection con- 
sists of a path in the network and some resources along 
each Ink m the path. In the case of a lightpath the re- 
source »s a wavelength. Two lightpaths that share a 
common fiber link must be assigned different wave- 
lengths Each lightpath can carry data at gigabit-per- 
socond ratos For example, in Fig. 1 lightpath 107 be- 
tween nodes 101 and 104 is established on wavelength 
>-2 lightpath 108 between nodes 102 and 105 is estab- 
lished on wavelength A^, and lightpath 109 between 
nodes 103 and 105 is established on wavelength A3 as 
well This model assumes that a lightpath must be es- 
tablished on a single common wavelength along its en- 
tire path. However if the network contains wavelength 
converter devices, it may be possible to assign a differ- 
ent wavelength on each link to the lightpath. 

In general the lightpaths terminate at a source and 
sink of data, for example electronic asynchronous- 
transfer-mode (ATM) switches. Depending on the re- 
quirements of the applications, such as the ATM switch- 
es, lightpaths may have to be setup and taken down 
over time. 

In order to establish or takedown a lightpath, the 
switches in the routing nodes along the path of the light- 
path must be configured. Further it must be noted that 
network elements such as the routing nodes, fibers be- 
tween them etc can fail. Thus it is desirable to perform 
this setup, takedown and updating the status of the net- 
work in a distributed manner. 

There are known distributed methods for setting up, 
maintaining and terminating connections in convention- 
al networks such as the internet [12], advanced peer-to- 
peer networks [3], ATM networks [9,11], or telecommu- 
nication networks (U.S. Patent 5,386,417 by Daugherty 
et al). 

Also known are specific centralized algorithms to 
determine a path and wavelength for a lightpath. For ex- 
ample, see [16, 8, 4, 2, 15, 5]. 

Prior art for setting up switches in a network remote- 



ly in response to messages is known. See for example, 
U.S. Patent 5, 1 89,41 4 by Tawara et al and the referenc- 
es below. 

The notion of multicasting messages to multiple 
destinations is well established in the literature. For ex- 
ample U.S. Patent 5,301,055 by Bagchi et al. discloses 
an algorithm that schedules time slots for multicast 
transmission inside a packet switch (as opposed to a 
network). Prior art teaches the notion of making rapid 
reservation for multiple network resources via a varia- 
tion on multicasting. For example, in the plaNET high- 
speed network ([9,10]), the controller at a source node 
wishing to reserve resources for a connection sends a 
message to the destination node along the path of the 
connection. This message is copied in hardware at the 
switches inside each intermediate node along the path 
and delivered to that node's controller. This hardware 
copy is very fast and allows this message to get through 
to all the controllers quickly. This technique cannot how- 
ever be applied to optical networks and to other net- 
works in general since the switches at the intermediate 
nodes do not have the hardware copy feature. 

It is an object of the invention to provide an im- 
proved communication network which uses a substan- 
tially different technique for rapid reservations. 

This object is met by the invention claimed in claim 

1. 

The preferred embodiment proposes distributed 
methods to setup, takedown and update lightpaths. 
These methods work reliably and are robust int he pres- 
ence of failures of various components in the network. 
The invention proposes the use of a controller associ- 
ated with each routing node that is responsible for con- 
figuring the switches in that routing node. The setup, 
takedown and update procedures are then accom- 
plished in a distributed manner by exchanging messag- 
es between the controllers according to specific proto- 
cols described in the embodiment. A rapid reservation 
procedure enables access to the resources needed to 
setup a lightpath efficiently and with high probability. The 
reservation procedure does not require the switches to 
have a copy feature, unlike the plaNET high-speed net- 
work [9,10]. Once a lightpath is setup it remains in place 
even if controllers at the intermediate node fail. A light- 
path may be taken down even if some controllers along 
the path of the lightpath are not operational. A controller 
upon recovering from failure obtains the correct status 
of the lightpaths flowing through its associated switch 
via the update procedure. The resources taken up by a 
lightpath are released when it is taken down by the take- 
down and update procedures. 

The preferred embodiment assumes that the light- 
paths terminate in ATM switches. It also assumes that 
controllers exchange messages between them, includ- 
ing the reservation and setup messages, using the ex- 
isting ATM protocols [6,1]. However the specific proto- 
cols proposed herein apply to networks having other ter- 
mination points besides ATM switches and to other 
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means of communication between the controllers as 
well. 

An embodiment of the invention will now be de- 
scribed, by way of example, with reference to the ac- 
companying drawings, in which: 

Fig. 1 describes the structure of an optical network 
consisting of optical wavelength-routing nodes intercon- 
nected by fiber links. 

Fig. 2 describes the structure of one node in the net- 
work. 

Fig. 3 shows the contents of a lightpath switch table 
at a given node. 

Fig. 4 shows the contents of a lightpath switch table 
at a given node after its controller fails and recovers. 

Fig. 5 explains the notion of permanent virtual paths 
and permanent virtual circuits. 

Fig. 6 describes the lightpath setup protocol. 

The System 

Fig. 2 illustrates an optical node (switching node) 
(101-105), consisting of the optical routing node (switch) 
201 , the associated ATM switch 202 with optical ports 
204A, 204B on it, and the controller 203. A similar node 
structure is discussed in [7]. The controller communi- 
cates with other controllers in the network through the 
ATM switch (202) and using ATM virtual circuits, to be 
described below. The node is connected to M other 
nodes via the set of fiber links 205. Each of these M links 
(106) carries 32 wavelengths (in general W wave- 
lengths). The optical routing node contains W - 1 inde- 
pendent optical switches (206 in figure), referred to as 
router switches, one for each wavelength except . The 
router switch that handles wavelength X will be referred 
to as the X router switch. The controller (203) is respon- 
sible for configuring the router switches (206). Wave- 
length X, is used for both control and data transmission 
and is treated differently than the others, as explained 
in the section on Control Connections. A wavelength de- 
multiplexor (207) separates the signals onto different 
spatial ports so that each wavelength is directed to a 
different port. The inputs to the X router switch are all 
the signals at wavelength X. on each of the M incoming 
links. Each output of the switch leads to a different wave- 
length multiplexor (208). This arrangement allows a sig- 
nal at a particular wavelength on an input port to be rout- 
ed to any output port, as long as no other signal at the 
same wavelength is being routed to the same output 
port. 

If the ATM switch has T input ports (204A) and T 
output ports (204B), the node can source Tlightpaths 
and sink Tlightpaths. Out of these, there are M prede- 
fined lightpaths, all at wavelength X-,, that will be dis- 
cussed in the section on Control Connections. The re- 
maining T- M sou reed lightpaths can be directed out on 
any of the A/f fiber links (205 A) connected to the node in 
any desired arrangement, provided that if two of these 
lightpaths share a link in the network they must be as- 



signed different wavelengths. A similar arrangement is 
true for the T- M incoming lightpaths. In order to realize 
these lightpaths, the node is provided with T-/Wtunable 
transmitters (209) and T- M receivers (210). Each tuna- 
s ble transmitter (209), capable of sending at one of the 
W - 1 wavelengths, is connected to an optical switch 
(211), referred to as a focal access switch, that is set to 
guide the signal to the desired router switch (206). Spe- 
cifically, if the transmitter connected to a local access 
switch is sending at wavelength X, the switch is set to 
guide the signal to the X router switch. On the receiving 
side, we have T - M receivers (210). Each receiver is 
connected to a local access optical switch (212), which 
selects a signal trombone of the router switches (206) to 
send to the receiver (210). Specifically, if the receiver 
(2 1 0) connected to a given port of the ATM switch is sup- 
posed to receive the ^ signal, the corresponding local 
access switch (212) is set to select the signal coming 
from the %± router switch (206). 

Three Levels of Connections 

In all-electronic ATM networks, two levels of con- 
nections are defined [6]: Virtual Path (VP) connections 
and Virtual Channel (VC) connections. The building 
blocks of the former are the physical links of the network, 
the building blocks of the latter are the VP's. ATM cells 
are sent on VC's, and several (typically many) VC's are 
bundled together into one VP. All VC's that use a partic- 
ular VP are packaged into the VP at one end, their cells 
travel together over the VP and at the other end of the 
VP, they are unpacked and forwarded separately to their 
respective next VP or to the user via the AAL layer. 

An ATM network with an underlying optical layer can 
be viewed similarly, except that there are three levels of 
connections: lightpaths, VP's and VC's. The building 
blocks of the lightpaths are the physical optical fiber 
links, the building blocks of the VP's are the lightpaths, 
and the building blocks of the VC's are the VP's. The 
lightpaths are switched by the optical routing node with- 
out being converted back into electronic form. All VP's 
that use a particular lightpath are packaged into the 
lightpath, and their cells travel optically over the light- 
path. At the other end of the lightpath, they are convert- 
ed to an electrical signal and forwarded together to the 
ATM switch. The ATM switch decides whether the cell 
should be forwarded to the next lightpath on the same 
VP or on a different VP. 

Lightpaths terminate at a port on the ATM switch, 
VP's terminate at the port of the ATM switch and VC's 
terminate at the user of the ATM switch. The fastest 
propagation of data through a node occurs if the signal 
travels on a lightpath, in which case it is handled and 
routed optically in the optical routing node. Next in speed 
of propagation through a node is a signal that arrives to 
the end of a lightpath, but continues on the same VP it 
has arrived on, on the next lightpath. In this case, the 
signal is converted to electrical and back to optical and 
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is handled by the ATM switch. Slightly slower propaga- 
tion occurs if the cell is at the end of the VP, and contin- 
ues on the same VC, but on a new VP. In this case, the 
signal is converted and handled by the ATM switch as 
in the previous case, but the switch needs to examine 
both the VP and the VC tables. In all previous cases, 
the controller is not involved in the transmission of data. 
Sometimes the controller must however be involved. 
For example if it must receive a control message, proc- 
ess it and transmit a new control message to the next 
controller on a path, the transfer of data is significantly 
slower than in other cases. Therefore in time critical pro- 
cedures, one should avoid involving intermediate con- 
trollers in the transmission. 

Lightpath Switch Table (LST) 

Every lightpath is identified by a lightpath-id that 
consists of a quadruple (originator, destination, wave- 
length, sequence number). Two different lightpaths may 
carry the same sequence number if they have different 
originators, destinations or wavelengths. The lightpath- 
id is assigned by the originator and we assume that the 
sequence number contains enough bits so that no wrap 
around occurs. This means that in our protocol we can 
assume that once a lightpath-id quadruple is assigned, 
it will never be reassigned to another lightpath ( in prac- 
tice, it may be reassigned a very long time after the pre- 
vious one was taken down, long enough to ensure that 
there is no trace of the old lightpath in the entire net- 
work). The lightpath-id is carried in all control messages 
related to the lightpath. 

The controller keeps a lightpath switch table (LST) 
that indicates the settings of its optical switches, the 
lightpath-id of the lightpaths that use them and their 
state. An example of such a table for wavelengths X 2 ,X 3 
at a node A whose neighbors are B, C, D, E, F, is given 
in Fig. 3. The entries correspond to: 

a ^ lightpath with id X, V,A^,12 (originator=X, des- 
tination=y, sequence number=12) that uses links 
(A, B) and (A, C). 

• a *2 lightpath with id A, Z ^,12 that uses Port 1 at 
the ATM switch at A and link (A, D). 

• is not in use on links (A, E)and (A F) and there- 
fore E and Fdo not appear in the link/port column 
for wavelength ^ 

a X 3 lightpath with id X, Z^tO that uses links (A, 
C) and (A F) that is being set up. 

• a X 3 lightpath with id V, 4 A3. 15 that uses link {A, D) 
and Port 2 of the ATM switch at A. 

• a lightpath with id A, U,X 3 ,A4 that uses Port 3 at 
A and link (A, E) that is being set up. 



• X3 is not in use on link (A, B). 

Definition 1 Related entries in the LSTs of two 
neighboring controllers are entries of the same wave- 

5 length that point to the link connecting them. For exam- 
ple, for two neighboring controllers A and B, an entryX^, 
(C, B) that points to link (A, B) for wavelength X% in the 
LST of controller A is related to an entry Xq, (A, D) in the 
LST at controller B. 

10 |f a controller goes down, it normally loses its LST 
When it comes back up, it initializes its table by reading 
the current switch settings in its optical routing node. It 
indudes in the LST in state up<B4> 1 those router switch 
ports that are connected to other ports. The LST will be 

is updated using a LST update protocol as described in 
the section on Takedown and Update. For example, 
consider the controller at node A with LST as in Fig. 3. 
As we shall see later, the router switches are set only 
when the lightpath goes to state up. For those in other 

20 states, the switches may be set according to old light- 
paths that have already been taken down. For example, 
suppose that Port 3 of the ATM switch is connected to 
link (A, F) on X 3 , link (A, C) is connected to link (A, 6) 
and link (A, E) is not connected. Also, on ,ink (A B) 

25 js connected to (A, C), Port 1 is connected to (A, D) and 
(A, E) is connected to (A f F). In this case, if the controller 
goes down, when it comes back up, it initializes the LST 
as in Fig. 5. 

When an optical routing node recovers from failure, 

30 its router switches are set randomly. In order to indicate 
to the controller (that might also be down or go down) - 
that the router switch settings were not set up by a pre- 
vious lightpath setup procedure, we use status bits. To 
every port of each router switch, we attach a status bit 

35 that is part of the optical routing node, but is reset by the 
controller. An optical routing node recovers from failure 
with all status bits set to 1 ( <A2> ). When a port is ac- 
tually connected to another port for the use of the light- 
path during the lightpath setup procedure to be de- 

40 scribed in the section on Setup, the controller resets the 
corresponding status bits of those ports to 0. A controller 
that recovers from failure disregards router switch set- 
tings that connect ports with status bits set to 1. The 
status bits are also used in case of partial failure of the 

45 optical wavelength router. If a router switch fails, it re- 
covers with status bits corresponding to its ports set to 1 . 

Control Connections 

50 Each ATM switch has ATM signaling capabilities im- 
plemented by a Signaling ATM Adaptation Layer (SAAL) 
assumed to work according to Draft ITU-T Recommen- 
dation Q.SAAL.O [1]. The control of the ATM switch, in 
particular setting up the VCI and VPI tables, is per- 
55 formed by the Signaling Network Layer in the ATM 
switch. To simplify the discussion here, we shall refer to 

1 The notation <•> refers to the corresponding line in the algorithm code 
of the section on The Code. 
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the Controller in the node to include both the Optical 
Routing Node Controller and the Signaling Network Lay- 
er. 

In order to set up, take down and manage light- 
paths, the controllers must communicate via messages 
such as reservation requests, setup messages. Com- 
munication in ATM takes place on VC's. This creates a 
chicken-and-egg problem: in order to set up lightpaths, 
the controllers need VC's, but in order to set up VC's we 
need VP's which in turn need lightpaths. The problem 
can be solved by using predefined lightpaths (P-light- 
path), predefined VP's (P-VP) and predefined Signaling 
VC's (PS-VC), similar to what was proposed in [11] for 
all-electrical ATM networks. On each link in the network, 
we shall predefine a P-lightpath, a P-VP and an S-VC, 
all of which are one-hop and are set up without commu- 
nication. The P-lightpath will carry the P-VP for that link, 
but can also carry other VP's ( the latter may consist of 
one or more links ). Similarly, the P-VP will carry the PS- 
VC, but may also carry additional VC's, for instance data 
transmission VC's. The PS-VC's allow controllers at 
neighboring nodes to communicate in order to set up 
other lightpaths. 

In order to realize the P-lightpaths, we select one 
wavelength in the entire network, say over which 
those lightpaths will be established, each at the time 
when the corresponding link is brought up. In the ATM 
switch, we designate M output ports and M input ports, 
where A/fisthe number of neighbors of the node, to serve 
the P-lightpaths (see Fig. 2). To each such port we con- 
nect a ^-fixed-wavelength transmitter or receiver that 
connects directly into the optical multiplexor/demulti- 
plexor that handles the corresponding fiber link. In this 
way we realize the P-lightpath without need for commu- 
nication between its endpoints. 

Routing of VP's and VC's is done by label swapping. 
The labels are normally referred to as identifiers, VPI 
and VCI respectively. Each port processor in the ATM 
switch contains a VPI table and a VCI table. The fields 
of an entry in the VPI table are (next- VPI, next-port). 
When a cell comes in, the processor looks in the entry 
corresponding to the VPI carried in the cell header, 
swaps the VPI with next-VPI and forwards the cell via 
the switch to next-port. A given next-VPI, say 0, is des- 
ignated to indicate that for this cell, the node is the end- 
point of a VP. When the port processor reads value 0 in 
the next-VPI field, it does not look in the next-port field, 
but rather examines the VCI table, in the entry corre- 
sponding to the VCI carried in the cell header. The fields 
of the VCI table are (next-VPI, next-VCI, next-port). The 
VPI, VCI in the cell header are swapped with next-VPI, 
next-VCI and the cell is forwarded to next-port via the 
switch! If the next-VPI field is 0 and the next-port field 
contains the port corresponding to the local higher level, 
it means that this is the endpoint of the VC. In this case, 
the next-VCI field indicates the local Adaptation ATM 
Layer (AAL) process that handles this call. 

Just as is preassignedforthe one-hop lightpath, 



we preassign the VPI-VCI pair (0,0) carried in the cell 
header to indicate that this cell belongs to the P-VP and 
PS-VC. Fig. 4 illustrates how P-VP(B^ C)an6 PS-VC 
(B -» C) ( corresponding to the link from B to C ) can be 

s established without communication. In Fig. 4, C B de- 
notes the port in the ATM switch at C that handles cells 
coming from the link (B,C) and B B denotes the port that 
handles cells coming from the local AAL. Every cell re- 
ceived by C on link (B,C) with VPI=0 and VCI=0 belongs 

10 by definition to P-VP(S -> C) and PS-VC(S-> C). The 
predefinition of P-VP(B C) and PS-VC(B -> C) re- 
quires only local operations at nodes B and C, without 
any communication between them. This is done in the 
following way. Node B writes 0 in the next-VPI field of 

is VP-Table(B s ), selects an available entry, 65 say, in VC- 
Table(5 e ), sets VC-Table(B 6 )[65] <- (0,0, B c ) and cre- 
ates an AAL process AAL(6g 5 ) for PS-VC (B -> C). In- 
dependently, node C writes 0 in the next-VPI field of VP- 
Table(C c ), selects an available entry, 98 say, in VC-Ta- 
20 ble(C c ), sets VC-Table(C c )[98] <- (0,0, C B ) and creates 
an AAL process AAL(C 98 ) for the same PS-VC. 

The Topology Update Protocol 

25 in order to set up new lightpaths, the controllers 
need to know the topology of the network and the usage 
of wavelengths on the network links. To this end, we em- 
ploy a topology update protocol, similar to the ones used 
in networks such as the ARPANET [13] or the plaNET 

30 high-speed network [9J. 

We will use a periodic topological update procedure 
in which each node periodically broadcasts pertinent to- 
pology information to ail the other nodes <B3>. The 
broadcast can be done either by flooding or on a span- 

35 ning tree if one is maintained for this purpose. We shall 
assume that there is some mechanism, either sequence 
numbers or timestamps, that provides ordering informa- 
tion about update messages originating at a node. 
The information in the topological update messages 

40 is generated from the node LST. A topological update 
message contains the list of up links adjacent to the 
node and their wavelength usage. The latter includes 
lightpaths that are up or being set up. For example, node 
A with the LST of Fig. 3 generates update message (B, 

45 Xa), (C, Xg, Xg), (ft A* \ 3 ), (£, X 3 ), (F f Ag). 

Every node in the network maintains a topology da- 
tabase that reflects its knowledge of the nodes that are 
up, the highest timestampof update messages received 
from each such node, the links that are up next to each 

50 node and their wavelength usage. Whenever a node re- 
ceives an update message about another node with a 
later timestamp than the stored one, it updates the cor- 
responding information and propagates it by sending 
the update message to all neighbors <E1>-<E4>. The 

55 topology information is used by nodes to decide the path 
of a new lightpath it intends to establish. 
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The Lightpath Update Procedure (LST Update) 

If a controller goes down, it loses its LST. After it 
comes back up, the status of the lightpaths that may go 
through its optical wavelength router must be updated, s 
To this end we employ a LST update procedure. Peri- 
odically (when UPDATE-TIMER expires) and whenever 
there is a change in the LST <C1>, each controller 
sends separately to each neighboring controller a list of 
the wavelength, id and state of all lightpaths that go over 
the link connecting them <C2>. For example, controller 
A with LST as in Fig. 3 sends to C the message ((^(X, 
V r ) X 2 ,12) l iyp) > (k 3 ,(X,Z,X3,10), reserved)). To neighbor- 
ing controller D it sends the message ((^.(A.Z,^,^), 
up),(X 3 ,( V.A^JSJ.up)). The actions taken by the 
neighboring controllers upon receiving the message will 
be described in the section on Takedown and Update. 

Setting Up Lightpaths 

The operations that need to be performed in order 
to establish a new lightpath between two nodes are the 
following: 

• Wavelength and Route Determination: The origina- 
tor node finds a path with an unused wavelength. 
This can be deduced from the topology database 
and wavelength usage database, both of which are 
updated by the topology update protocol. Path com- 
putation algorithms that can be used for this pur- 
pose are described in [8, 1 5). 

• Reservation: The originator node requests all con- 
trollers on the lightpath to reserve the selected 
wavelength on all links of the path. This stage is 
necessary because the number of wavelengths is 
limited, say 32, and without it there might be several 
independent attempts to use the same wavelength 
on some link, that will collide and none will be suc- 
cessful. In order to reduce the chance of collision of 
the reservation processes, this stage must be per- 
formed at maximum speed. 

• Lightpath setup/ release: If the reservation step is 
successful at all nodes in the path, router switches 
are set by the corresponding controllers to accom- 
plish the light path. Otherwise, the reservation is re- 
leased at all nodes at which it was accomplished. 
Router switch settings do not change upon reser- 
vation or release. 

We propose the following reservation protocol (see 
Fig. 6). The controller (203A) at the originator node 
(node A) establishes a VC to each of the controllers 
(203B, 203C, 203D) of the nodes of the lightpath, a sep- 
arate VC to each controller (see Fig. 6(a)). The VC's 
(601 B, 601 C.601 D) are established over the P-VP's (not 
shown) We shall use the term reservation- VC's io refer 



to these VC's. On these VC's the originator can perform 
a preliminary checkio find out whether indeed the wave- 
length is still available on the selected path. After ob- 
taining whatever information it needs, the originator con- 
troller triggers the reservation procedure, by sending a 
RESERVE message <G6> for the wavelength on the in- 
tended path, simultaneously on all reservation -VC's 
(channels). Note that although the setup of the reserva- 
tion-VC's and the gathering of information may take 
some time, the RESERVE messages travel on the res- 
ervation- VC's and do not use interim controllers. In this 
way, we minimize the probability that this process will 
collide with another reservation process that may at- 
tempt to reserve the same wavelength on one or more 
links of the path. Each controller on the path sends a 
positive (RESERVE-ACK) or negative (RESERVE- 
NACK) response to the originator <H5>,<H8>. If the res- 
ervation is successful, a corresponding entry is included 
in the LST in state reserved <H6>. No router switch set- 
tings are changed at this point, since there is no point 
in setting the switches unless it is known that all reser- 
vations were successful. The controller uses the RE- 
SERVE-TIMER <H4> in state reserved and deletes the 
entry if the timer expires before it receives the SETUP 
message <N2>. 

If the reservation was successful at all nodes, then 
the setup procedure, described below (see Fig. 6(b)), is 
started. The reservation may not be successful at all 
nodes even if the wavelength was available on all links 
at the time when the preliminary check was performed. 
This may happen if some of the controllers have in the 
interim received and accepted another reservation re- 
quest for the same wavelength on one or more of the 
links of the path. In this case, the originator instructs all 
controllers, by TAKEDOWN messages sent over the 
reservation-VC's <J2>, to release the reservation. 

The setup procedure cannot be performed on the 
reservation-VC's because loops of reserved wave- 
lengths may occur that will never be released (see Lem- 
ma 2). Another reason that the setup must be performed 
sequentially is that the last controller on the path must 
be the last to setup the switches. This is because once 
the last controller on the path receives the setup mes- 
sage, the node is allowed to start sending data on this 
lightpath. Therefore, if the reservation procedure was 
successful at all nodes, the originator starts a chain of 
SETUP messages (602) <I3> (see Fig 6(b)) : that travel 
from one controller to the next on the path, on the PS- 
VC's. When a controller receives a SETUP message 
<K1 >, if the LST contains an entry in state reserved with 
the same lightpath-id <K2>, it sets up the router switch 
for the appropriate wavelength to be used by the light- 
path, updates the status bits as explained below, puts 
the entry into state up and sends a SETUP message to 
the next controller on the path. 

To be specific, consider the controller at node A that 
is supposed to connect ports B and C at the X router 
switch for use of the lightpath. Suppose that beforehand 
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port B is connected to port D and port C to port E, both 
connections being unused. Setting the router switch 
means connecting ports B and C. At the same time, 
ports D and E will be connected to some other unused 
ports in the switch, possibly to each other. 

Definition 2 The connection between ports B and 
C above will be referred to as a connection between sol- 
idly connected ports. The connection between ports D 
and E above will be referred to as a connection between 
spuriously connected ports. 

When the router switch is set up, the status bits cor- 
responding to solidly connected ports are reset to 0 and 
the status bits corresponding to spuriously connected 
ports are set to 1 (<I6>, <17>, <K5>, <K6>). 

If the SETUP message makes it all the way to the 
destination controller, the latter sends a REPLY mes- 
sage to the originator on the reservation-VC <K9>. The 
originator node and destination node on the path can 
start using the lightpath when they receive respectively 
the REPLY and the SETUP messages. 

Takedown and Update 

If a lightpath is not needed anymore or if a link or 
an optical routing node on the lightpath fails, the light- 
path should be taken down and its resources released. 
A lightpath can be taken down intentionally only if there 
are no VP's, and therefore no VC's : that use that light- 
path. From the control point of view, taking down a light- 
path means updating the LST's at all controllers along 
the lightpath by deleting the corresponding entry. The 
router switch settings in the optical routing node remain 
the same until they are needed for a new lightpath. 

We employ two procedures for taking lightpaths 
down, explicit TAKEDOWN messages and LST update- 
When there are no exceptions, either one will take the 
entire lightpath down. Each procedure handles a differ- 
ent scenario of exceptions. 

The first procedure calls for the originator controller 
to send a TAKEDOWN message on each reservation- 
VC to each controller on the lightpath <P1>. Upon re- 
ceiving the TAKEDOWN message, each controller de- 
letes the entry corresponding to the lightpath in its LST 
<R2>. The property of this procedure is that, even if 
some controllers on the lightpath are down, it takes 
down the lightpath at all working controllers that are con- 
nected to the originator. However, if the originator is 
down, the lightpath cannot be taken down by this pro- 
cedure. Also, if a link or an optical routing node is down, 
there is no reservation-VC to controllers beyond the 
failed link, so again at these controllers the lightpath 
cannot be taken down by TAKEDOWN messages. 

The second procedure is LST update, whereby pe- 
riodically (when UPDATE-TIMER expires) and whenev- 
er there is a change in the LST <C1>, each controller 
sends separately to each neighboring controller a list of 
the wavelength, id and state of all lightpaths that go over 
the link connecting them <C2>. When a controller re- 



ceives the LST update message from a neighbor, it 
scans the entries in the local LST that point towards the 
link to that neighbor and looks for related entries in the 
received LST. If for a given entry in the local LST there 
s is no related entry in the received one, the entry in the 
local LST is deleted <D4>. 

The LST update procedure has the property that if 
any controller on the lightpath deletes the entry corre- 
sponding to a lightpath, the appropriate entries will be 
10 deleted at all controllers along the lightpath until the pro- 
cedure meets a failed controller or the end of the light- 
path. If a link or an optical routing node fails, both ends 
of the lightpath will detect toss of light and will delete the 
entry in the LST at the controllers. In addition, the con- 
ts trollers next to the failure will delete the entry corre- 
sponding to all lightpaths that have traversed the failed 
link/router <F2>. Then the LST update procedure will 
ensure that the entries corresponding to the lightpath 
will be deleted along the lightpath. Similarly, if the orig- 
inator controller fails and the other end wants to take 
down the lightpath, it will simply delete the appropriate 
entry. 

Note that since transmission of data on a lightpath 
does not employ the controller or the ATM switch at in- 
termediate nodes, failure of a controller or of an ATM 
switch after the lightpath is established does not require 
taking down the lightpath. Since the LST is lost upon the 
failure of the controller, when the controller comes up 
again, it assumes that the router switch settings con- 
necting ports with status bit = 0, correspond to lightpaths 
that are up <B4>, <B5>. If the lightpath is in fact not up, 
the controller will learn about this via the LST update 
procedure. 

In order to deal with the case when all controllers 
along a lightpath fail and then recover, the endpoints of 
a lightpath have a timer (IDLE-TIMER) that runs while 
receiving idle characters; if the timer expires, the entry 
corresponding to the lightpath is deleted <02>, thereby 
triggering the deletion of entries at all other controllers 
via the LST update procedure. 

While in reserved state, the controller does not 
change the state of the entry in response to received 
LST updates <D3>. This prevents a controller from tak- 
ing the reservation down in the case when it has re- 
served a wavelength while its neighbor is slow and has 
not done so yet. The RESERVE-TIMER runs in reserved 
state to ensure that the reservation is released if the 
lightpath is not set up within reasonable time. The expi- 
ration time of the RESERVE -TIMER should be set such 
that it allows the reservation phase to finish at all nodes 
on one hand, while it allows cancelation if the setup 
process does not complete successfully within reason- 
able time. 

An additional case that has to be studied is when 
there is a failure of a controller while a lightpath is in 
reserved state at that controller. In this case, the light- 
path will not be set up and the reservations must be re- 
leased. This is done again by the RESERVE-TIMER ex- 
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piling at controllers that do not receive the SETUP mes- 
sage <N2>. Also all entries corresponding to a link are 
deleted if the link or adjacent optical routing node fail 
<F2>. The LST update procedure ensures in both cases 
that the entries will be deleted along the lightpath. s 

The Code 

The following code, reflecting the previous informal 
description of the protocol, is executed by each control- 10 
ler in the network. 

Protocol Manage 

Messages is 

LST - Lightpath Switch Table - each entry contains 
(wavelength, id, state) 
SETUP(ld, path) 

TAKEDOWN(id) 20 
RESERVE(id, X) 

Variables 

id - lightpath - id 25 

LST - lightpath switch table 

state(entry) - state of an entry 

state(id) - state of an entry with a given lightpath-id 

entry(id) - entry with a given id 

status bit - for each port in the router and local ac- 30 
cess switches 



Initialization 

setup P-lightpaths, P-VPs and PS-VCs 
initialize LST to empty 

initialize topology database entries to list of adja- 
cent links and no reserved lightpaths. 
start UPDATE-TIMER 



Algorithm for each controller 

Actions related to all lightpaths 



35 



40 



event: optical wavelength router comes up 45 
all status bits = 1 
delete all entries in LST 
event: controller comes up 

corresponding to each switch setting 

so 



A1 

A2 
A3 
B1 
B2 

with status bit = 0 
B3 send local topology update 

B4 enter entry in LST 

B5 state(entry) = up ; id(entry) = 

empty 

B6 if switch setting points to local 55 

port: start IDLE-TIMER(entry) 
C1 event: UPDATE-TI MER expires or change 

in LST 



C2 send LST update 

C3 start UPDATE-TI MER 

D1 event: LST received from adjacent controller 
D2 for each entry x in local LST that 

points to the link to the adjacent controller 
03 if state(x) = up and received 

LST does not contain related entry 
D4 delete entry x 

E1 event: topology update received 
E2 if timestamp higher than stored one 

E3 update link utilization informa- 

tion 

E4 forward topology update 

F 1 event: loss of light on P-lightpath (adjacent 

link or optical wavelength router fails) 

F2 delete all entries in LST that use the 

link 

Actions related to a given lightpath id 

G1 event: lightpath setup request from higher 
layer (originator) 

G2 determine route and wavelength X 

G3 setup reservation-VC's with each 

controller along route 

G4 rf all reservation-VC's are success- 

fully set up 

G5 assign entry and id 

G6 send RESERVE(idA) on each 

reservation-VC 

G7 state(id) = reserved 

G8 start RESERVE-TIMER(id) 

H1 event: RESERVE(id, X) received 

H2 jf X available on corresponding link 

(no entry in LST) 

H3 select entry and record id 

H4 start RESERVE-TIMER(id) 

H5 send RESERVE -ACK(id) 

H6 state (id) = reserved 

H7 else 

H8 send RESERVE-NACK(id) 

1 1 event: RESERVE -ACK(id)'s received (orig- 
inator) 

12 il all RESERVE-ACK(id)'s received 
and state(id) = reserved 

13 pass SETUP(id) to the next 
controller 

14 state (id) = up 

15 set router and local access 
switches 

16 status bits corresponding to sol- 
idly connected switch ports = 0 

17 status bits corresponding to 
spuriously connected switch ports ~ 1 

18 start SETUP-TIMER(id) 

19 cancel RESERVE-TIMER(id) 
J1 event: RESERVE-NACK(id) received 

J2 send TAKE DOWN (id) messages 
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over reservation-VC's 

J3 delete entry(id) messages over res- 

ervation-VC's 

K1 event: SETUP(id) received 
K2 if state (id) = reserved 5 

K3 state(id) = up 

K4 set router switch 

K5 status bits corresponding to 

solidly connected switch ports = 0 
K6 status bits corresponding to io 

spuriously connected switch ports = 1 
K7 if destination 

K8 set local access switch 

K9 send REPLY(id) to origi- 

nator over reservation-VC is 
K10 start IDLE-TIMER(id) 

K1 1 stan using lightpath 

K12 else 

K1 3 pass SETUP(id) to next 

controller 20 

K14 cancel RESERVE-TIMER(id) 

L1 event: REPLY(id) received ( originator ) 

L2 start using lightpath 

L3 cancel SETUP-TIMER(id) 

L4 start IDLE-TlMER(id) 25 

M1 event: message received on lightpath(id) 

M2 restart IDLE-TIMER(id) 

N1 event: RESERVE -Tl ME R(id) expires 

N2 delete entry(id) 

N3 originator: send TAKEDOWN (id) 30 

messages over reservation-VC's 

01 event: IDLE-TIMER(id) expires (originator 
or destination) 

02 delete entry(id) 

03 originator: send TAKE DOWN (id) 35 
messages over reservation-VC's 

P1 event: lightpath takedown request from 

higher layer ( originator or destination ) 
P2 delete entry(id) 

P3 originator: send TAKE DOWN (id) 40 

messages over reservation-VC's 
Q1 event: loss of light on lightpath(id) (origi- 
nator or destination) 
Q2 delete entry(id) 

Q3 originator: send TAKE DOWN (id) 45 

messages over reservation-VC's 
R1 event: TAKEDOWN(id) message received 
R2 delete entry(id) 



Definition 3 

a) A router switch setting is said to belong to the 
lightpath for which the setting was set 

b) An entry in state up is said to correspond to the 
router switch setting that has been set when the en- 
try entered up (<I5>, <K4> ) or to the router switch 
setting for which the entry was entered in the LST 
when the controller came up ( <B4> ). A router 
switch setting is also said to correspond to the 
switch ports that are connected by the switch set- 
ting. 

Lemma 1 (Preliminary Properties) Consider a given 
lightpath with a given id and a given selected route. For 
the given lightpath 

a) at most one RESERVE(id) message arrives at 
each controller. 

b) each controller can be in reserved state at most 
once. 

c) at most one SETUP(id) message arrives at each 
controller 

d) an entry in state up always contains the correct 
corresponding router switch setting. 

e) if an entry is in state up , the status bit at the cor- 
responding switch ports is 0. 

Theorem 1 (Successful setup) Suppose that a con- 
troller receives a lightpath setup request from the higher 
layer. Suppose that a route is found and no failures of 
links, switches or controllers occur along the route for 
sufficiently long time. Suppose that in addition no other 
originator reserves the same wavelength on one of the 
links of the route before the considered controller does. 
Then: 

a) the wavelength is successfully reserved at all 
controllers along the lightpath (state(id)= reserved). 

b) if, in addition no RESERVE-TIMER expires, the 
lightpath is set up at all controllers along the light- 
path (state(id) = up). 



Properties of the Control Procedures 

Below is a set of theorems that demonstrate the re- 
liable, failsafe operation of the invention. Recall that we 
have assumed earlier that the lightpath-id is unique and 
no two lightpaths carry the same id. 



so C ) if in addition, the SETUP-TIMER at the originator 

does not expire, both the originator and the desti- 
nation will start using the lightpath. 

Definition 4 

55 

a) A linear lightpath section is a path of router 
switches on a given wavelength X at nodes A lt A^ t 
A n such that: 
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• the X router switch at A; connects A h1 toA i±1 for 
i=2, ~, (n-1)^ 

• the X router switch at A 1 connects A 2 either to 
the local access switch at A 1 or to some node 
A 0 whose X router switch is not connected to 
the X router switch at A 

• the X router switch at A n connects A n . A either to 
the focal access switch at A n or to some con- 
troller A n+1 whose X switch is not connected to 

Ar 

b) A loop lightpath section is the same as a linear 
one, ezcept that the X router switch at A 1 connects 
A 2 to A n and the X router switch at A n connects A^ 
toA v 

Lemma 2 (Loop Freedom) There are no loop light- 
path sections with all corresponding status bits = 0. 

Corollary: There are no loop lightpath sections with 
corresponding LST entries in state up . 

A possible alternative may be to send the SETUP 
message in parallel on the reservation- VC's instead of 
sequentially from controller to controller. With this alter- 
native however, it controllers fail and recover, it is easy 
to construct a case where a loop with status bits = 0 can 
be formed with all controllers in state up . If such a loop 
is formed then it will never be broken. 

Definition 5 A lightpath is said to be up if it has been 
set up successfully and its endpoints are using it 

Theorem 2 If a lightpath is up, the corresponding 
entries at all controllers that are alive are in state up. 

Definition 6 

a) Two controllers on some lightpath section for 
some wavelength X are said to Reconnected // the 
optical switches and the links between them on the 
lightpath are up. 

b) Two controllers on some lightpath section for 
some wavelength X are said to strongly connected 
if they are connected and the controllers between 
them on the lightpath are also up. 

In the following we assume that the UPDATE -TIM- 
ER is such that the frequency of LST updates is high 
compared to lightpath setup attempts. As seen in the 
proof of Lemma 3b), if this is not the case, there may be 
race conditions between the two that will prevent re- 
lease of resources. 

Lemma 3 

a) A controller cannot stay indefinitely in state 
reserved . 



If A m has the entry corresponding to the considered 
section in state up and A p does not have a corre- 
sponding entry, then A m will also delete the entry. 

Theorem 3 (Re/ease of resources) Suppose that 
the topological changes on the route of a lightpath have 
ceased and that the lightpath is not up. Then: 

a) The corresponding entry is deleted at all control- 
lers that do not receive the SETUP message and at 
all controllers that are strongly connected to one of 
the controllers that do not receive SETUP. 

b) If the originator controller is up , the correspond- 
ing entry will be deleted at all controllers that are 
connected to it. 

c) If the destination controller is up, the correspond- 
ing entry will be deleted at all controllers that are 
strongly connected to it 

d) If a link or an optical routing node is down, the 
corresponding entry will be deleted at all controllers 
that are strongly connected to the controllers that 
are adjacent to the failed link/router. 

Extensions 

The use of predefined lightpaths on each link re- 
30 quires use of a wavelength (X,) for a one-link lightpath 
on each link and M ports of the ATM switch dedicated 
to these lightpaths, where Mis the degree of the node. 
Although the predefined lightpaths may be used for both 
control and data, there may not be sufficient wave- 
35 lengths or ATM switch ports to accommodate them. In 
this case, one may work with a spanning tree or a ring 
for control messages. However this will decrease the re- 
liability for control purposes. Also, maintaining the tree 
or ring in the event of failures requires complicated pro- 
40 tocols, whereas our approach using the predefined one- 
link lightpaths on all links requires no maintenance at 
all. For instance, it is not dear how one can rebuild a 
spanning tree if one or more of the branches of the tree 
fails, without communication between the separated 
45 tree portions. 

While our protocols ensure that there are no loop 
lightpaths with the status bits all set to 0, it is possible 
that there may still be loops with some of the router 
switches in the loop having their status bits set to 1 . This 
50 may lead to undesirable effects at the physical layer 
such as saturation of the optical amplifiers (if any) in the 
loop. However, preventing such a phenomenon requires 
additional hardware at each switch port that disconnects 
the link corresponding to an unused port from the switch. 

55 



b) Let A m and A p be two controllers that are strongly 
connected and remain so for sufficiently long time. 
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Claims 

1. A communications network having a plurality of 
switching nodes interconnected by a plurality of 
links, each of said switching nodes having a switch 
and an associated controller which controls the con- 
figuration of said switch, and connection means for 
establishing a desired connection between an orig- 
inating switching node of said switching nodes and 
a destination node of said switching nodes, said 
connection means comprising: 

(a) means for computing a path comprising of 
a set of said links interconnecting a set of said 
nodes needed to establish said desired con- 
nection from said originating node to said des- 
tination node; and 

(b) means for sending reservation requests si- 
multaneously through separate channels from 
one of said controllers to each node of said set 
of nodes interconnected by said route to re- 
serve and determine if said set of links are 
available to establish said desired connection 
along said route; and 

(c) means for, upon receiving an acknowledge- 
ment that said set of links for said connection 
are available and have been reserved, trans- 
mitting a setup message sequentially between 
adjacent nodes starting from said originating 
node to each node of said set of nodes along 
said path until said setup message reaches 
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said destination node, where said setup mes- 
sage is transmitted only between two adjacent 
nodes at any given time, and where each node 
of said set of nodes changes its switch config- 
uration in response to said setup message so s 
as to interconnect said set of links to establish 
said desired connection. 

2. A communications network as claimed in claim 1 
wherein the switching nodes are optical switching io 
nodes and the links are optical fibers. 

3. A communications network as claimed in claim 1, 
the connection means further includes: 

is 

means for storing a switch table at each of said 
switching nodes, each entry of said table iden- 
tifying a connection through a switch of said 
each node, and said each entry indicating the 
stale of said connection, where said state indi- 20 
cates whether said connection is currently es- 
tablished or reserved for future use; and 

means for updating each said switch table of 
each node of said set of nodes to indicate that 2s 
corresponding links of said set of links are re- 
served for future use to establish said desired 
connection, each node of said set of nodes be- 
ing updated in response to receiving one of said 
reservation requests; 30 

wherein each controller of each node of said 
set of nodes changes the state of said desired 
connection in said switching table from re- 
served to a state indicating current use of said 3S 
desired connection where each said controller 
of each node of said set of nodes changes its 
switch configuration in response to said setup 
message so as to interconnect said set of links 
to establish said desired connection. 40 



45 
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